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Abstract 

We study the fermionic couplings of the neutral Higgs bosons in the THDM, 
assuming a four-texture structure for the Yukawa matrices. We then de- 
rive the low-energy constraints on the model, focusing in b-quark and lepton 
physics, and apply them to study Higgs boson detection at future colliders. 
We show that the bound on the flavor-violating parameter Xsb obtained from 
the contribution due to the fts/i'^-coupling to the decay 5 — > s -|- 7 (roughly of 
the order 10~^ — 10"^) is approximately a factor 10 more restrictive than that 
obtained from the current bound on T{B'^ fi~ fi'^) (which gives a bound on 
Xsb of the order 10'^ — 10^^), while LFV decay /i ^ 67 constraints x^tr ^ 10^^. 
These constraints imply that a future muon collider could be able to detect 
Higgs boson signals from the decays fi'^r^ and bs for tan (3 < 15, 

while such signals turn out to be too small for tan /3 > 20. At a hadron collider 
it is further possible to study the Higgs boson coupling h^bb, by searching for 

the associated production of the Higgs boson with bb pairs. 
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I. INTRODUCTION. 



Despite the success of the Standard Model (SM) in the gauge and fermion sectors, the 
Higgs sector remains the least tested aspect of the model, and the mechanism of electroweak 
symmetry breaking (EWSB) is still a puzzle. However, the analysis of raditive corrections 
within the SM [1] , points towards the existence of a Higgs boson with mass of the order of 
the EW scale, which could be detected in the early stages of LHC [2]. On the other hand, 
the SM is often considered as an effective theory, valid up to an energy scale of 0{TeV), and 
eventually it will be replaced by a more fundamental theory, which will explain, among other 
things, the physics behind EWSB and perhaps even the origin of flavor. Several examples 
of candidate theories, which range from supersymmetry [3] to deconstruction [4], include a 
Higgs sector with two scalar doublets, which has a rich structure and predicts interesting 
phenomenology [5] . The general two- higgs doublet model (THDM) has a potential problem 
with flavor changing neutral currents (FCNC) mediated by the Higgs bosons, which arises 
when each quark type (u and d) is allowed to couple to both Higgs doublets, and FCNC 
could be induced at large rates that may jeopardize the model. The possible solutions to 
this problem of the THDM involve an assumption about the Yukawa structure of the model. 
To discuss them it is convenient to refer to the Yukawa lagrangian, which is written for the 
quarks fields as follows: 

where $1,2 = {4>i,2^ 4>i,2)^ denote the Higgs doublets. The specific choices for the Yukawa 
matrices Yi^2 {q = ^) d) define the versions of the THDM known as 1, II or III, which involve 
the following mechanisms, that are aimed either to eliminate the otherwise unbearable FCNC 
problem or at least to keep it under control, namely: 

1. DISCRETE SYMMETRIES. A discrete symmetry can be invoked to allow a given 
fermion type (u or d-quarks for instance) to couple to a single Higgs doublet, and in 
such case FCNC are absent at tree-level. In particular, when a single Higgs field gives 
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masses to both types of quarks (either — Yf — or Y2 — Y2 — 0), the resulting 
model is referred as THDM-I. On the other hand, when each type of quark couples 
to a different Higgs doublet (either Y^ = Y2 = Q or Y2 = Yf = 0), the model is 
known as the THDM-II. This THDM-II pattern is highly motivated because it arises 
at tree-level in the minimal SUSY extension for the SM (MSSM) [5]. 

2. RADIATIVE SUPRESSION. When each fermion type couples to both Higgs doublets, 
FCNC could be kept under control if there exists a hierarchy between Y'"''^ and Y2''^- 
Namely, a given set of Yukawa matrices is present at tree-level, but the other ones arise 
only as a radiative effect. This occurs for instance in the MSSM, where the type-II 
THDM structure is not protected by any symmetry, and is transformed into a type-III 
THDM (see bellow), through the loop effects of sfermions and gauginos. Namely, the 
Yukawa couphngs that are already present at tree- level in the MSSM (Y^, Y2) receive 
radiative corrections, while the terms (1^2'^, F") are induced at one-loop level. 

In particular, when the "seesaw" mechanism [6] is implemented in the MSSM to explain 
the observed neutrino masses [7,8], lepton flavor violation (LFV) appears naturally 
in the right-handed neutrino sector, which is then communicated to the sleptons and 
from there to the charged leptons and Higgs sector. These corrections allow the neutral 
Higgs bosons to mediate LFV, in particular it was found that the (Higgs-mediated) 
tau decay r ^ 3/x [9] as well as the (real) Higgs boson decay H ^ t/j, [10], can enter 
into possible detection domain. Similar effects are known to arise in the quark sector, 
for instance B ^ iifi can reach branching fractions at large tan /3, that can be probed 
at Run II of the Tevatron [11,12]. 

3. FLAVOR SYMMETRIES. Suppression for FCNC can also be achived when a certain 
form of the Yukawa matrices that reproduce the observed fermion masses and mixing 
angles is implemented in the model, which is then named as THDM-III. This could be 
done either by implementing the Frogart-Nielsen mechanism to generate the fermion 
mass hierarchies [13], or by studying a certain ansatz for the fermion mass matrices 
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[14] . The first proposal for the Higgs boson couphngs along these fines was posed in 
[15,16], it was based on the six-texture form of the mass matrices, namely: 



Ml 



^ Cg ^ 

c; B, 
V a,j 



Then, by assuming that each Yukawa matrix Y^^^ has the same hierarchy, one finds: 
Aq ~ mqg, Bq ~ .^Jm^^ffT^ and Cq ~ ^mqjriq^. Then, the fermion-fermion'-Higgs 
boson (//V*^) couplings obey the following pattern: Hfifj ~ ^nif.mf./mw, which is 
known as the Cheng-Sher ansatz. This brings under control the FCNC problem, and 
it has been extensively studied in the literature to search for fiavor- violating signals in 
the Higgs sector [17]. 

In this paper we are interested in studying the THDM-III. However, the six-texture 
ansatz seems disfavored by current data on the CKM mixing angles. More recently, mass 
matrices with four-texture ansatz have been considered, and are found in better agreement 
with the observed data [18,19]. It is interesting then to investigate how the Cheng-Sher form 
of the ff'4>^ couplings, get modified when one replaces the six-texture matrices by the four- 
texture ansatz. This paper is aimed precisely to study this question; we want to derive the 
form of the couplings and to discuss how and when the resulting predictions could be 

tested, both in rare quark and lepton decays and in the phenomenology of the Higgs bosons 
[10]. Unlike previous studies, we keep in our analysis the effect of the complex phases, which 
modify the FCNC Higgs boson couplings. 

The organization of the paper goes as follows: In section 2, we discuss the lagrangian for 
the THDM with the four-texture form for the mass matrices, and present the results for the 
//'(/)° vertices in the quark sector. Then, in section 3 we study the constraints impossed on 
the parameters of the model from low energy fiavor violating processes. Section 4 includes 
the predictions of the model for both, fiavor conserving (PC) and fiavor violating (FV) Higgs 
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boson decays. While in section 5, we discuss the capabihties of future ii^ji and hadron 
coUiders to detect such decays. Finally, section 6 contains our conclusions. 



II. THE QUARK SECTOR OF THE THDM-III WITH FOUR-TEXTURE MASS 

MATRICES 

The Yukawa lagrangian of the THDM-III is written for the quarks fields as follows: 

= Y^Q^^^UR + Y^Ql^2Ur + Y^QL^^dR + Y^QL^2dR (2) 

where $i_2 = (0^2; 0i,2)^ denote the Higgs doublets. The specific choices for the Yukawa 
matrices Y]^2 {q — d) define the versions of the THDM known as I, II or III. 
After spontaneous symmetry breaking the quark mass matrix is given by, 

M,^^{v,Y^ + V2Y^), (3) 

We will asuume that both Yukawa matrices Yi and Y2 have the four-texture form and 
Hermitic; following the conventions of [18], the quark mass matrix is then written as: 

Cq ^ 



when Bq ^ one recovers the six-texture form. We also consider the hierarchy: 

I ^4^ I I I, I I, I Cq I, which is supported by the observed fermion masses in the SM. 

Because of the hermicity condition, both Bg and Ag are real parameters, while the phases 
of Cq and Bq, ^Bq,Cq, can be removed from the mass matrix Mq by defining: Mq = P^MqPq, 
where Pq — diag[l, e^'^^" , e^^'^^i'^^'^''^], and the mass matrix Mq includes only the real parts 
of Mq. The diagonahzation of Mq is then obtained by an orthogonal matrix Oq, such that 
the diagonal mass matrix is: Mq = O^MqOq. 

The lagrangian (2) can be expanded in terms of the mass-eigenstates for the neutral 
(h'^jH^jA^) and charged Higgs bosons (H^). The interactions of the neutral Higgs bosons 
with the d-type and u-type are given by {u, u' — u, c, t. and d,d' — d,s, b.), 
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(4) 



The first term, proportional to Sqq> corresponds to the modification of the THDM-II over the 
SM result, while the term proportional to Y2 denotes the new contribution from THDM-III. 
Thus, the ff'4>^ couplings respect CP-invariance, despite the fact that the Yukawa matrices 
include complex phases; this follows because of the Hermiticity conditions impossed on both 
Yi« and Yl 

The corrections to the quark flavor conserving (FC) and flavor violating (FV) couplings, 
depend on the rotated matrix: Y2 — OqPqY2P^Oq. We will evaluate Y2 assuming that Y2 
has a four-texture form, namely: 



^ CI 0^ 
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(5) 



The matrix that diagonalizes the real matrix Mq with the four-texture form, is given by: 



^^V (A^-A^)(A^-A^) 
, A?(A-A|)(A^-A|) 



(A^-A^)(A^A^) 

A^(A,-A^)(A^-A,) /A|(A^-A|)04^-A|) 



Al(Ag-A,) 
(A^-A^)(Ag-A^) 



A^{Xl-X\){Xl-Xl) V A^{Xl-X\){Xl-Xl) ) 



where =| Af |, ml 



A? 



\2 I, ^3 

Then the rotated form Y2 has the general form. 



A3 I, and Tjq — Al/ml (g = u,d). With = m". 
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2 J 11 (^2') 12 (^2*^)13 



V 



(i"2'')21 (^2^)22 (i"2'')23 • (6) 
(^*)31 (^^)32 (^*)33 

However, the full expressions for the resulting elements have a complicated form, as it 
can be appreciated, for instance, by looking at the element (^2 )22, which is displayed here: 
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(7) 



''3 ~ ^2 

where we have taken the hmits: |74g|, mf, m2 » m\. The free-parameters are: Sf, Bl, A^, Aq. 

To derive a better suited approximation, we will consider the elements of the Yukawa 
matrix as having the same hierarchy as the full mass matrix, namely: 



q q q 

Bl = hl^{Aq-Xl){ml-Aq) 
Bl^hl{ml-Aq + Xl) 



A2 — a\Aq. 



(8) 

(9) 
(10) 

(11) 



Then, in order to keep the same hierarchy for the elements of the mass matrix, we find 
that Aq must fall within the interval (777.3 — ^V) ^ A.q< m\. Thus, we propose the following 



relation for Aq. 



Aq = ml{l-pqZq), 

where Zq — ml/ ml <^ 1 and < Pq < 1. 

Then, we introduce the matrix as follows: 



(12) 



which differs from the usual Cheng-Sher ansatz not only because of the appearence of the 



complex phases, but also in the form of the real parts 



1-3 



Expanding in powers of Zq^ one finds that the elements of the matrix x'^ have the following 
general expressions: 



xl,-{al-ble-^^-^)r^,^, 

Xl2 = kv, + K + bl- ibfe^"-^ + ble-'^-^m 



~q q 
X33 = «2 



(14) 



While the diagonal elements x% ^'^^ I'sal, we notice (Eqs. 14) the appearance of the 
phases in the off-diagonal elements, which are essentially unconstrained by present low- 
energy phenomena. As we will see next, these phases modify the pattern of flavor violation 
in the Higgs sector. For instance, while the Cheng-Sher ansatz predicts that the LFV 
couplings (^2^)13 and (I2 )23 vanish when = case this is no longer valid for 

cos^Bq 7^ 1- Furthermore the LFV couphngs satisfy several relations, such as: \x2z\ — IXish 
which simplifies the parameter analysis. 

In order to perform our phenomenological study we find convenient to rewrite the la- 
grangian given in Eq. (4) in terms of the Xqq' — Xij ^ follows: 
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(15) 



where u, u' = u, c, t. and d,d' = d,s,b., and unlike the Cheng-Sher ansatz, Xqq' {q q') 
complex. 

Finally, for completeness we display here the corresponding lagrangian for the charged 
lepton sector, which has been already reported in our previous work [20], namely 
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where Z, I' — e, /i, r. 

On the other hand, one can also relate our results with the SUSY-induced THDM-llI, 
for instance by considering the effective Lagrangian for the couplings of the charged leptons 
to the neutral Higgs fields, namely: 



-C = LLYilR(t>l + LiXi (eil + esFj-y.) lR(t>l* + h.c. 



(17) 



In this language, LFV results from our inability to simultaneously diagonalize the term Yi 
and the non-holomorphic loop corrections, t2YiY},Yy. Thus, since the charged lepton masses 
cannot be diagonalized in the same basis as their Higgs boson couplings, this will allow 
neutral Higgs bosons to mediate LFV processes with rates proportional to e|. In terms of 
our previous notation we have: Y2 = ^2YiYjYy. Thus, our result will cover (for some specific 
choices of parameters) the general expectations for the corrections arising in the MSSM. 



III. BOUNDS ON THE FLAVOR VIOLATING HIGGS PARAMETERS 

Constrains on the FV-Higgs interaction can be obtained by studying FV transitions. In 
this section we consider the radiative decay 6^57 and the decay B^^ A*~A*^) which 
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together with LFV bounds derived in [20] constraint the parameter space of THDM-III, and 
determine possible Higgs boson signals that may be detected at future colliders. 

3.1 Radiative decay 6 — > S7. We will make an estimation of the contribution due to the 

flavor- violating ff'4>^ couplings to the standard model branching ratio of 6 — > s 7 as follows 

A5r(6^ 57) = Ar(6^ 57) X I ^ T{b ^ clTJi)] (18) 

\l=e,n,T I 

Such contribution to the branching ratio of 6 ^ 57 at one loop level is then given by [21] 



ttemm^mf cos^(a - /3) 2 
^^'^"^ ^'^^^ 16vrKo|K.P cos^/3 

cos (a — /?) 
X -smaH Xhh 



m^o 2 



2 



(19) 



Prom Eqs. (14) we have Xsh — Xdb — li^-i ~ ^2^~^^'^'')WPd- We will make use of the good 
agreement between the current experimental value for Br{b s^) = (3.3 ± 0.4) x 10~^ 
and the theoretical value obtained for Br{b —>■ sj) = (3.29 ± 0.33) x 10""^ in the context 
of the standard model [22] to constraint any new contribution to Br{b 57), namely 
ABr{b — > 57) ^ 10~^, and hence to bound Xsb{— Xdb) ^ ^ function of m^o, x^b, ol and 
tan p. 

a) Assuming m/jO = 120 GeV and Xbb — 0, we depict in Fig. 1 the values of the upper bound 

on Xsh {{Xsbf^b,"') as a function of tan /3, for a = /3, /3 — 7r/4, /9 — 7r/3. 

b) Taking a = /5 — 7r/4 and Xbb = 0, we plot in Fig. 2 the results for {Xsbf^h.'^ as a function 
of tan /3, for m^o = 80 GeV, 120 GeV, 160 GeV. 

c) We show in Fig. 3, taking m^jo = 120 GeV and a — P — n/A, our numerical results for 
{Xsb)'^y^b.^ as a function of real values ^ of Xbb for tan/3 = 5, 25, 50. 

From Figs. 1-3, we conclude that the upper bound on the LFV parameter Xsb, from the 
radiative decay 6^5-1-7 measurements, is much more restrictive for large values of tan /3, 



^We will study the dependency on the phases {xfj = xjj^ '■') of the Higgs phenomenology in 
a forthcoming paper. 
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Xbb ~ — 1, ruho 80 GeV and a ^ (3 . However, one can still say that at the present time 
the coupling Xsb is not highly constrained when tan /3 ~ 5 — 10, or even for larger values of 
tan/3 provided that Xbb — >^+lorQ;— — 7r/2, thus Xsb could induce interesting direct LFV 
Higgs boson signals at future coUiders. 

3.2 Bg — > //"//"^ decay. The formula to calculate the width of the decay — > at 
the one loop level is given as follows [11] 



where Gp = 1.16639-^ GeT^-^ r]^^^-.^ ^ 1.5, tub c^5GeV, and fe = 180 MeV. 

We will make use of the current experimental limit for r(5° — > < 8.7 x 10~^^ 

GeV [11,23] to constraint the LFV parameter Xsb{— Xdb) and the resulting upper bound will 
be shown as function of rUhO, Xnin and tan/3. 

a) Assuming m/jO = 120 GeV and = 0, we depict in Fig. 4 the values of the upper 
bound on Xsb {{Xsb)u"b^'^'^) as a function of tan/3, for a = /3, /3 — 7r/4, /3 — 7r/3. 

b) Taking a — P — n/A and Xufi — 0, we plot in Fig. 5 the results for {Xsb)u^b^^^ ^ ^ function 
of tan/3, for m^o = 80 GeV, 120 GeV, 160 GeV. 

c) We show in Fig. 6, taking m^jo = 120 GeV and a — /3 — 7r/4, our numerical results for 
{Xsb)u"C^'^ as a function of real values ^ of Xnii for tan/3 = 5, 25, 50. 

From Figs. 4-6, we conclude that the upper bound on the LFV parameter Xsb, obtained 
from the experimental bound for the width of the radiative decay B^ fJ'~IJ''^, is more 
restrictive for large values of tan/3, ~ —1, m^o fa 80 GeV and a ~ /3. However, one 
can still say again that at the present time the coupling Xsb is not highly constrained for 
tan /3 ~ 5 — 10, or even larger values of tan /3 provided that X/i/x — >■ +1 or a — >• /3 — 7r/2. 

From Eqs. (18) and (19), we obtain the following relation 



Vqcd ml fl nis rub ml cos^ja - (5) 
1287rm^„ cos^ /3 



cos(q; — 0) ^ 
X -smaH j^— x^,^, 



(20) 
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Assuming that Xtifi = Xbb (or ~ 10 ^ and Xbb ^ 10 ^) and taking ABr{b ^ 57) < 
10~^, which is a conservative bound [22], we get 



6.7 X 10-21 Gey for rriho = 80 GeF 

4.8 X 10-21 GeV for m^o = 120 Gel^ (22) 
3.8 X 10-21 GeV for m^o = 160 GeV 



Thus, we conclude from (22) that the bound on the parameter Xsb obtained from the 
constraint on the contribution due to the 6s/i°-couphng to the theoretical branching ratio 
of the radiative decay 6 — > s + 7 is approximately a factor ten more restrictive than that 
one obtained from the current experimental bound for r(S° — > ii'/i'^) already mentioned 
[11,23]. 

IV. HIGGS BOSON DECAYS IN THE THDM-III 

One of the distinctive characteristic of the SM Higgs boson is the fact that its couphng 
to other particle is proportional to the mass of that particle, which in turn determines the 
search strategies proposed so far to detect it at future colliders. In particular, the decay 
pattern of the Higgs boson is dominated by the heaviest particle allowed to appear in its 
decay products. When one considers extensions of the SM it is important to study possible 
deviations from the SM decay pattern as it could provide a method to discriminate among 
the different models [24]. 

Within the context of the THDM-III, which we have been studying, not only modification 
of the Higgs boson couplings are predicted, but also the appearance of new channels with 
flavor violation, both in the quark and leptonic sectors [10,25]. 

To explore the characteristics of Higgs boson decays in the THDM-III, we will focus 
on the hghtest CP-even state (/i°), which could be detected first at LHC. The fight Higgs 
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boson-fermion couplings are given by Eqs. (15) and (16), where we have separated the SM 
from the corrections that appear in a THDM-III. In fact, we have also separated the factors 
that arise in the THDM-III too. We notice that the correction to the SM result, depends 
on tan/3, a (the mixing angle in the neutral CP-even Higgs sector) and the factors Xij that 
induce FCNC transitions (for i ^ j) and further corrections to the SM vertex. 

In what follows, we will include the decay widths for all the modes that are allowed 
kinematically for a Higgs boson with a mass in the range SOGeF < m/jO < 160 GeF. 
Namely, we study the branching ratios for the decays hP bb, cc, tt, iiJI and the flavor- 
violating bs{sb), t]I{iit), as well as the decays into pairs of gauge bosons with one real 
an the other one virtual, i.e. /i° WW*, ZZ*. 

Making use of Eqs. (15) and (16) we obtain 
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r(/i° ^ II) = 
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(27) 



,77N _ g^mfeo mi mi, cos^(a-/3) / A(mf , mf,, m^„) 2 



(28) 



where: X{x, y,z) — {x — y — z)"^ — 4yz; u, u' — u, c, t; d, d' — d , s ,b; and I, I' — e~, /i", t~ . 
For the decays /i° — > WW*, ZZ* we use the corresponding expressions given in Ref. [2]. 

Wc calculate the branching ratios for all the relevant decay modes that are allowed 
kinematically in the range 80GeV < m^o < 160 GeV; taking a — P — 37r/8; assuming 
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Xij — 0.1 for i — j and i ^ j. We consider the following cases tan/3 = 2, 2.61, 5, 15 and 50. 
Our results are displayed in Figs. 7-11, where we notice the important effect that the factor 
Xbb has on the mode — > bb, which could be dominant for certain range of parameters, 
but it could be suppressed for other choices. Fig. 12 clarifies what is going on, it shows the 
region in the plane {a — P) - tan/5, where the coupling h'^bb vanishes, and one can notice 
that this happens even for small values of the parameter Xbb (~ 0.01). 

We also notice in Figs. 7-11 that the Br for the FCNC mode — > bs(bs) reaches values 
above 10"^ and the LFV mode hP — > tJi{t(i) reaches values above 10~^ for 5 < tan /3 < 50 
and 80 GeV < m^o < 155 GeV . Further, in the mass range when 'Qx{hP —>■ bb) is not domi- 
nant, we find that the modes hP — > VFl^*, ZZ* become the dominant ones. 

Overall, our results show that the usual search strategies to look for the SM Higgs boson 
in this mass range, may need to be modified in order to cover the full parameter space of 
the THDM-III. 

In the coming sections we will discuss how the Higgs boson signals could be searched 
at a future /i.+/i~-collider. We will also study the reach in parameter space that could be 
obtained through the Higgs boson production in association with a pair of 6-quarks at LHC, 
which was found to be relevant in the large tan P limit for the MSSM [26] . 

V. PROBING THE FERMIONIC HIGGS BOSON COUPLINGS AT FUTURE 

COLLIDERS 

In order to probe the Higgs vertices we will consider first the search for the LFV Higgs 
boson decays at future muon colliders, which was proposed some time ago [27], namely we 
will evaluate the reaction ^ /i° ^ /'/"■ Then we will consider the production of 

Higgs bosons at the LHC, to probe both LFV and hPbb couphngs. 

5.1 Tests of LFV/FCNC Higgs boson couplings at /j.'/j,'^ -colliders. An option to 

search for LFV couplings, could be provided by the reaction: /^^(pa) — >■ 0" — >■ 

f'iPc) + f'ipd)- The s-channel Higgs boson cross section (on resonance) is given by: 
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where 0° denotes a neutral Higgs boson which decays to a final state /'/"• The effective 
cross section ct<^o is obtained by convoluting with the Gaussian distribution in [28]: 

a A, , ^ / n - ^npm^ (^°' 

^ tot 



(7^ can be expressed in terms of the root-mean-square (rms) Gaussian spread of the energy 
of an individual beam, i?, as follows: 

In this work, we will restrict our numerical analysis to the case of the light neutral scalar 
i.e. 0° = and for the most relevant cases /'/" = T l^^ij^t^~)-i hs{hs). 

The calculation of ct^jO requires the evaluation of the following quantities: T{h^ r"//"*"), 
r(/i° ^ 6s), r(/iO ^ and r^^^", which are given in Eqs. (22)-(27). 

By performing a detailed numerical analysis one can show that ^ 

0.98 < l^^' Z\^p < 1.02 (32) 

provided that: 80 Gel/ < ruho < 160 Gel/; -7r/3 < a - < 0; 5 < tan/? < 50; \x^^\ < 0.01. 
Hence, under the previous conditions we have 



^ g^m,,o ml sin' a / ^ 

32 TT cos^ /9 y m^o / 

Assuming 80 GeT/ < niho < 160 GeT/, we can write 

r£ = E r(^° ^ /T) + E r(/i° ^ /T) + r(/i° ^ w^w-*) + r(/i° ^ zz*), (34) 
/' /',/" 



^We will discuss the dependency on the parameters Xij s-iid the phases 'd^j (Xij — Xij^^^^) of the 
decay widths r((/>° — > fifj) for = h^, H^, and A° in a forthcoming paper. 
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where /', /" ^ i-quark. 

It is also possible to show numerically that ^ 

0.98 < < 1.06, (35) 

provided that the following conditions are satisfied: — 7r/3 <« — /9<0;5< tan/3 < 
~ 0.01; |x/'/"| ^1 (/' 7^ /")• Hence, under the previous conditions we can 
approximate 

rS*^rSlx,v"=o- (36) 

We can write the cross-sections of the processes nT t~ and \i~ —>■ bs as 
follows: 

tot 

, , , Att Br(h^ ^ i2+ij-)Br(h^ ^bs) 



where 



todx/'/"=0 

r(/iO ^ r-^+) 



tot lx/'/"=0 



Brih^^bs)..^^!^^^ (39) 

tot lx/'/"=0 



provided that |x/'/"l ^ 10"' for /' = /"; and |x/'/"| < 1 for /' ^ 

We will calculate the number of events r^/x+ (r+/i^) produced in a /x^/x """-collider 

^^^-^rM ^ ^^o(^-^+ ^ r-^+) X Lyear. (40) 

Then our numerical results for N'^^^'~''^^(s — m^o, Xht) are shown in Figs. 13-22, as a function 
of tan/3, by taking: (i) = 1 (Figs. 13-17) and (ii) x^t = {XtiT)u7b^'^ ^ the value of the 
upper bound on x^t obtained from the experimental measurement of the radiative decay 
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— > e+7 [20], we will take the current experimental result Br{ii^ — > < 1.2 x 10 
[22] (Figs. 18-22). We plot curves for mfeo = 80 GeV , 120 GeV , 160 GeF, taking a = (5, 
jS — Tr/i, /5 — 7r/3, assuming yearly integrated luminosities I/^ear = 0.1, 0.22, I fb~^ for beam 
energy resolutions of = 0.003%, 0.01%, 0.1%, respectively [27]. 

Prom Figs. 13-17, we could expect the production of ~ 10^ — 10^ T~/x"'"(r"'"//~) pairs with 
a "'"-collider. However, if we calculate the number of such events using the constraint 
on XfMT obtained from the experimental bound on the branching ratio of the LFV process 
/x"*" 7, the production rates are drastically reduced, specially for large values of tan /3 

( > 15), as it can be observed in Figs. 18-22. We can conclude that the detection of r~ fi'^ 
or T+//~ events would be possible for tan/3 < 15, but not for tan/3 > 15. 

On the other hand, the nonobservation of at least an event t" (or r"*"//"") in a year 
would imply that 

N^^-^^{s = mlo,x,r)<l, (41) 
which would also allow us to put an upper bound on x^^, namely: 

{X,rrj:r'{s = mlo) = [n^^-^>^{s = mlo,x,r = 1)]"'^' (42) 

According to Figs. 18-22, the n^n^ collider measurements could improve the bound on x^r 
obtained from the radiative decay //"*" e"'"7, {XtJ-T)u7b^'^ ^ o^^X tan/3 < 15. 

Then, for the quark signals, we will calculate the number of events bs (bs) produced in 
a /i^/x+-collider, given by: 

^MM^fes ^ ^ 55) X Lyear- (43) 

We depict our numerical results for N^^^''''^{s = Tn\o,Xsb) in Figs. 23-32, as a function 
of tan/3, by taking: i) Xsb — 1 (FigS- 23-27) and ii) Xsb — {Xsb)^xU}.'^ i the value for the 
upper bound on Xsb obtained in subsection 3.1 from the good agreement between the ex- 
perimental and theoretical value of the radiative decay 6 — > 57 (Figs. 28-32). We plot 
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curves for rriho — 80 GeV, 120 GeV, 160 GeV, taking a — P, P — 7r/4, P — n/3, assum- 
ing yearly integrated luminosities I/^ear — 0.1, 0.22, I fb~^ for beam energy resolutions of 
R = 0.003%, 0.01%, 0.1%, respectively [27]. 

Prom Figs. 23-27, we would expect the production of ~ 10^ — 10^ bs(bs) pairs at a 
/i~/i"'"-collider. However, the number of such events obtained by using the constraint on 
Xsb impossed by the branching ratio of the process b ^ sj, are drastically reduced for 
tan/3 > 15, as it can be observed in Figs. 28-32. Again, we can conclude that the detection 
of 6s or 6s events would be possible for tan/3 < 15, but not for tan/3 > 15. 

Similarly, the nonobservation of at least an event of the type 6s (or 6s) in a year, could 
be used to improve the bound on Xsb obtained from the radiative decay b ^ s^, {Xsbj'^b.'^ ^ 
only if tan/3 < 15. 

5.2 Search for Higgs boson in associated production with 6-quarks pairs at LHC. 

The associated production of the Higgs boson in association with a quark pair 66, has been 
found useful to detect the neutral Higgs bosons of the MSSM [5] , especially in the large-tan /3 
domain. Here we will show that this reaction can be also useful to constrain the coupling 
h%b in the THDM-III. 

As shown in Ref. [26], the reaction pp — > > 66) + bb + X produces a large sample 
of events which could be detectable provided a i^-factor is above a certain value, which 
depends on the Higgs boson mass and the coupling h^bb (which enter in the event rate both 
from the Higgs boson production and decay), this factor is defined as 



To have a detectable signal at LHC for ruho — 150 GeV, the modulus of this factor has to 
be above \K\min = 1-93, as obtained from a detailed analysis of signal and backgrounds 
performed in Ref. [26], to which we refer for details of kinematical cuts, acceptances and 
parton distributions. 

In Figs. 33-35, we show the region of the plane tan/3 - (a — /3), where the signal for 
m^o = 150 GeV^ is detectable. One can notice that the effect of the parameter Xbb, even 




(44) 
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for small values, can have a dramatic impact on the extension of the region of parameters 
where the signal is detectable. Therefore, LHC will be able to constrain the presence of 
a non-minimal flavor structure (which is reflected on the parameters Xij)^ provide a 
decisive test of the fermionic coupling of the Higgs boson. 

5.3 Search for LFV Higgs boson decays at Hadron colliders. We will concentrate 
here on the LFV Higgs boson decays (pi —>■ t/j,, which has a very small branching ratio within 
the context of the SM with light neutrinos ( < 10~^ — 10~^ ), so that this channel becomes 
an excellent window for probing new physics [10,29,30]. The decay width for the procces 
4>i — > T/j, (adding both final states t~^ij,~ and r"/^"'" ) can be written in terms of the decay 
width r{Hi — > tt), as follows: 



Therefore, the Higgs boson branching ratio can be approximated as: Br{(f)i — > t/i) — 
{R$nY X Br{(j)i — > tt). We calculated the branching fraction for h — > t//, and find that it 
reaches values of order 10~^ in the THDM-lIl; for comparison, we notice that in the MSSM 
case, even for large values of tan/3, one only gets Br{h Tji) ~ 10~^. 

These values of the branching ratio enter into the domain of detectabihty at hadron 
colliders (LHC), provided that the cross-section for Higgs boson production were of order of 
the SM one. Large values of tan (3 are also associated with large b-quark Yukawa coupling, 
which in turn can produce and enhancement on the Higgs boson production cross-sections 
at hadron colliders, even for the heavier states and either by gluon fusion or in the 
associated production of the Higgs boson with b-quark pairs; some values are shown in table 
1; these were obtained using HIGLU [31]. Thus, even the heavy Higgs bosons of the model 
could be detected through this LFV mode. 



r(0,^T//) = {Rt\^V{Hi^TT) 



(45) 



where 




(46) 
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mH,A [GeV] 




< [Pb] 


all [pb] (~ a^,) 


150 


126.4 (492.6) 


129.1 (525.) 


200 (800) 


200 


29.5 (114.3) 


29.1 (120.) 


100 (400) 


300 


3.6 (13.5) 


3.15 (13.6) 


20 (80) 


350 


1.6 (5.9) 


1.2 (5.6) 


12 (48) 


400 


0.75 (2.75) 


0.73 (2.8) 


8 (32) 



Table 1. Cross-section for Higgs boson production at LHC, through gluon fusion (c^'"^) and 
in association with bb quarks, {cr^'^), for tan/3 = 30 (60). 

For instance, for mH,A — 150 GeV and tan P — 30(60) the cross-section through gluon 
fusion at LHC is about 126.4 (492.6) pb [31], then with Br{H r^) ~ lO^^^lO^^) and an 
integrated luminosity of 10^ pb~^, LHC can produce about 10^(10^) LFV Higgs boson events. 
In Ref. [32] it was proposed a series of cuts to reconstruct the hadronic and electronic tau 
decays from h ^ Tfi and separate the signal from the backgrounds, which are dominated by 
Drell-Yan tau pair and WW pair production. According to these studies [32], even SM-like 
cross sections and m^f, ~ 150 GeV, one coud detect at LHC the LFV Higgs boson decays 
with a branching ratio of order 8 x 10~^, which means that our signal is clearly detectable. 

VI. CONCLUSIONS 

We have studied in this paper the couplings that arise in the THDM-HI, using 

a Hermitic four-texture form for the fermionic Yukawa matrix. Because of this, although 
the couplings are complex, the CP- properties of h^, (even) and ^4^^ (odd) remmain 

valid. 

Wc have derived bounds on the parameters of the model, using current experimental 
bounds on LFV and FCNC transitions. One can say that the present bounds on the couplings 
Xij 's still allow the possibihty to study interesting direct flavor violating Higgs boson signals 
at future colliders, provided one takes not too large values of tan/3 ( < 15). 
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In particular, the LFV couplings of the neutral Higgs bosons, can lead to new discovery 
signatures of the Higgs boson itself. For instance, the branching fraction for h'^ tJ1{t^) 
can be as large as 10~^, while Br{h — > bs(bs)) is also about 10~^. These LFV Higgs modes 
complement the modes S° /i/i, r — > S/i, t ^ /I'j and /i — > 67, as probes of flavor violation 
in the THDM-HI, which could provide key insights into the form of the Yukawa mass matrix. 

Thus, the coming generation of colliders will provide a decisive test of the Yukawa sector 
of the SM and its extensions, as well as other properties of the gauge-Higgs sector [33] 
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Figure Captions 

Fig. 1: The upper bound {Xsbj'^y^b.'^ as a function of tan/3, ior a — (3, a — (3 — 7r/4, 
P- 7r/3, with ABr{b ^ 57) < 10-^ taking m^o = 120 GeV and = 0. 

Fig. 2: The upper bound {XsbTy^b,'^ as a function of tan/3, for m/jO = 80 GeV, 120 GeF, 
160 Gel/, with ABr{b ^ S7) < 10-^ taking a = /3 - 7r/4 and Xbb = 0. 

Fig. 3: The upper bound {Xsbjti^b.'^ as a function of Xbb, for tan/3 = 5, 25, 50, with 
ASr(6 ^ 57) < 10-^ taking m,,o = 120 GeV and a = /3 - 7r/4. 

Fig. 4: The upper bound {Xsb)u"C^'^ ^ ^ function of tan/3, for a = /3, a — /3 — 7r/4, 
CK = /3 - 7r/3, with r(SO ^ ^-^+) < 8.7 x lO-^^ Q^y^ taking m^o = 120 GeV and x,^m = 0- 

Fig. 5: The upper bound {Xsb)u"b^'^'^ as a function of tan/3, for ruho — 80 GeV, 120 GeF, 
160 Gel/, with r(5° ^ < 8.7 x lO^^^ GeV, taking a = /3 - 7r/4 and x,.,. = 0. 

Fig. 6: The upper bound {Xsb)u"b^'^'^ as a function of Xnn: for tan/3 = 5, 25, 50, with 
r(S° ^ < 8.7 X 10-19 GeV, taking m^o = 120 GeV and a = /3 - 7r/4. 

Fig. 7: Branching ratios for all the relevant decay modes that are allowed kinematically 
for SO GeV < m/jo < 160 Gel/; taking a — P — Sn/S with tan/3 = 2; assuming Xij — 0.1 for 

i = j and i 7^ j. 

Fig. 8: Same as in Fig. 7, but for tan/3 = 2.61 
Fig. 9: Same as in Fig. 7, but for tan /3 = 5 
Fig. 10: Same as in Fig. 7, but for tan /3 = 15 
Fig. 11: Same as in Fig. 7, but for tan/3 = 50 

Fig. 12: Curves in the plane (a — /3) - tan/3 in which the coupling bbh^ vanishes, for 
Xbb = 0.01, 0.1, 0.5, and 1. 
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Fig. 13: Number of events N''^''^''^' as a function of tan /5; taking Xht — 

1, for s = mlo = 

(120 Gey)^, a — and yearly integrated luminosities Ly^ar — 0.1, 0.22, 1 fb~^ and beam 
energy resolutions of = 0.003%, 0.01%, 0.1%, respectively. 

Fig. 14: Number of events N^^^'^^ as a function of tan/?; taking x^it = 1) for s = mlo = 
(120 GeV^)^, a — (3 — 7r/4, and yearly integrated luminosities Lyear — 0.1, 0.22, 1 fb~^ and 
beam energy resolutions of i? = 0.003%, 0.01%, 0.1%, respectively. 

Fig. 15: Number of events TV'^m-^tm ^ function of tan/3; taking x^t = 1, for s = m^o = 
(120 GeV^)^, a = P — tt/3, and yearly integrated luminosities Lyear = 0.1, 0.22, I fb~^ and 
beam energy resolutions of i? 0.003%, 0.01%, 0.1%, respectively. 

Fig. 16: Number of events N^^~''^^ as a function of tan/3; taking Xt^r = 1, for s = m^o = 
(80Gey)^, a — P — 7r/4, and yearly integrated luminosities Ly^ar — 0.1, 0.22, 1 fb~^ and 
beam energy resolutions of i? = 0.003%, 0.01%, 0.1%, respectively. 

Fig. 17: Number of events N'^^^^'^^^ as a function of tan/3; taking XiiT — 

1, for s = m^o = 

(160 GeV)^, a = /? — 7r/4, and yearly integrated luminosities Ly^ar = 0.1, 0.22, 1 fb~^ and 
beam energy resolutions oi R — 0.003%, 0.01%, 0.1%, respectively. 

Fig. 18: Number of events N^^~''^^ as a function of tan/3; taking Xht — {Xht)uj,!'^ with 
Br{ii'^ e+7) < 1.2 x 10"-^^ for s = m^o = (120 GeF)^, a = /3, and yearly integrated lumi- 
nosities Lyear = 0.1, 0.22, 1 and beam energy resolutions of i? = 0.003%, 0.01%, 0.1%, 
respectively. 

Fig. 19: Number of events N'^^^'^^ as a function of tan/3; taking Xi^t = {XiJ.T)u7b^^ 
with Br(ii+ e+7) < 1.2 x 10-^^ for s = m^o = (120^6^)2, a = /3 - 7r/4, and 
yearly integrated luminosities I/^ear = 0.1, 0.22, 1 fb~^ and beam energy resolutions of 
R = 0.003%, 0.01%, 0.1%, respectively. 

Fig. 20: Number of events N^^~''^^ as a function of tan/3; taking Xij.t = {Xiir)'^'^'^ 
with Br{^J+ e+7) < 1.2 x 10-^^ for s = m^o = (120Ge\/)2, « = /3 - 7r/3, and 
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yearly integrated luminosities Ly^ar — 0.1, 0.22, 1 fb ^ and beam energy resolutions of 
R = 0.003%, 0.01%, 0.1%, respectively 

Fig. 21: Number of events N^^'~''^^ as a function of tan/3; taking Xfir — {XtiT)u'2^'^ 
with Brin+ e+7) < 1.2 x 10"", for s = m^,, = {SOGeVf, a = (3 - 7r/4, and 
yearly integrated luminosities Ly^ar — 0.1, 0.22, 1 fb~^ and beam energy resolutions of 
R = 0.003%, 0.01%, 0.1%, respectively. 

Fig. 22: Number of events N^^'^'^^ as a function of tan/3; taking Xij-t — {XnT)u7b^'^ 
with Br{fi+ e+7) < 1.2 x 10-^\ for s = m^o = {160GeVy, a = (5 - 7r/4, and 
yearly integrated luminosities Ly^ar = 0.1, 0.22, 1 fb~^ and beam energy resolutions of 
R = 0.003%, 0.01%, 0.1%, respectively. 

Fig. 23: Number of events N^^^~*^^ as a function of tan/3; taking Xsb — 1, for s — m^o = 
(120 Gey)^, a = /3, and yearly integrated luminosities Ly^ar — 0.1, 0.22, 1 fb~^ and beam 
energy resolutions of i? = 0.003%, 0.01%, 0.1%, respectively. 

Fig. 24: Number of events iV'^'^^^* as a function of tan/3; taking Xsb = 1, for s = "^^o = 
(120 GeV^)^, q; = /3 — 7r/4, and yearly integrated luminosities Lyear — 0.1, 0.22, 1 fb~^ and 
beam energy resolutions of i? = 0.003%, 0.01%, 0.1%, respectively. 

Fig. 25: Number of events N^^^""^^ as a function of tan/3; taking Xsb = 1, for s = m^o = 
{120GeVy, a — P — 7r/3, and yearly integrated luminosities Lyear = 0.1, 0.22, 1 fb~^ and 
beam energy resolutions of i? = 0.003%, 0.01%, 0.1%, respectively. 

Fig. 26: Number of events N^f^^^^ as a function of tan/3; taking Xsb = 1) for s = m^o = 
{SOGeVy, a — P — 7r/4, and yearly integrated luminosities Lyear — 0.1, 0.22, 1 fb~^ and 
beam energy resolutions of it! = 0.003%, 0.01%, 0.1%, respectively. 

Fig. 27: Number of events N^^^~*^^ as a function of tan/3; taking Xsb — 1, for s — m\o — 
(160 GeV)^, a = j3 — 7r/4, and yearly integrated luminosities Lyear = 0.1, 0.22, I fb^^ and 
beam energy resolutions oi R — 0.003%, 0.01%, 0.1%, respectively. 
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Fig. 28: Number of events N^'^'^^^ as a function of tan/3; taking Xsb — {Xsb)y^b,'^ with 
ABr{b s^) < 10~^, for s = m\a = (120 GeF)^, a = (3, and yearly integrated lumi- 
nosities Lyf.ar = 0.1, 0.22, 1 fh~^ and beam energy resolutions of i? = 0.003%, 0.01%, 0.1%, 
respectively. 

Fig. 29: Number of events N'^'^'^^^ as a function of tan/3; taking Xsb — {Xsb)''y^b.'^ with 
ABr{b s^) < 10~^, for s — m\o — (120 GeF)^, a — (5 — 7r/4, and yearly integrated lumi- 
nosities Lyear = 0.1, 0.22, 1 fh~^ and beam energy resolutions of i? = 0.003%, 0.01%, 0.1%, 
respectively. 

Fig. 30: Number of events N^^^'^^ as a function of tan/3; taking Xsb — {Xsb)^-!^^.^ with 
ABr{b sj) < 10~^, for s — m\o — (120 GeF)^, a = /3 — 7r/3, and yearly integrated lumi- 
nosities Lyf.ar = 0.1, 0.22, 1 fb~^ and beam energy resolutions of i? = 0.003%, 0.01%, 0.1%, 
respectively. 

Fig. 31: Number of events N'^'^^'^'^ as a function of tan/3; taking Xsb = {Xsb)'^uM^ with 
ABr{b s^) < 10~^, for s — m\o — (SOGeF)^, a — (5 — 7r/4, and yearly integrated lumi- 
nosities Lyear — 0.1, 0.22, 1 fb~^ and beam energy resolutions of R — 0.003%, 0.01%, 0.1%, 
respectively. 

Fig. 32: Number of events N'^'^^''^ as a function of tan/3; taking Xsb = (Xs6)u^*^ with 
ABr{b S'-f) < 10~^, for s — m^o — (160 Gel/)^, a — (3 — 7r/4, and yearly integrated lumi- 
nosities Lyear — 0.1, 0.22, 1 fb~^ and beam energy resolutions of i? = 0.003%, 0.01%, 0.1%, 
respectively. 

Fig. 33: \K\ as a function of tan/3; taking ruho = 150 GeV, a — (3 — Sn/S. Assuming 
Xij = 0.1 for i ^ j and: (a) Xu = 0-01 (line A); (b) Xu = 0.1 (line B); (c) Xu = 0-5 (line C); 
(d) Xu = 1 (line D). 

Fig. 34: Same as in Fig. 33, but for a = /3 - 7r/4 
Fig. 35: Same as in Fig. 33, but for a = /3 — 7r/8 
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